BD 155 213 

AUTHOfi 

TITiE 



PUB.DSTB €. 
NOTE 



EDRS PRICE . 
DESCRIPIORS 



IDENTIFIERS 



/ 



DOCUHEHT BESUHE 



V. 



TH 007 216 



Slaughter, Helen B* ; Gallas, IdwinJiJ^ 
Hill Out-of Level Nora-fief efenced Testing Improve the 
Selection of Prograk Participants and the Diagnosis 
of Reading Coaprehension in ESEA litle I Prograas? 
Bar 78 . ' 

26p*; Paper presented at the Annual Meeting of the 
American Educational Research Association (62nd, 
Toronto, Ontario, Canada; March 27-31, 1978) ; For 
related documents, see TH 007 212 and 215 

I!F-$0.83 flC-$2.06'pius Postage. 

Achievement Tests; Coaparative Testing ; Disadvantaged 
Youth; Equated Scores; Grade 6; ♦Grad^ Equivalent 
Scores; Intermediate Girades; ♦Low Achievers; Norm ' 
Referenced Tests; ♦Noras; Ran Scores; Reading 
Achieve^aent; Reading Difficulty; Scores; Standardized 
Tests; lest Bias; ♦Testing; ♦Testing Probleas; ^Tes't 
Interpretation; Test Reliability 

California Achievement Tests; Elementary Secondary 
' Education Act Title I; ♦Out ct Level Testing 



ABSTRACI 

Concer.n was 
testing Elementary Secondary 
with norm-referenced tests t 
students wi.ll have scores in 
students obtaining equal sea 
easier cut--of-level tests va 
sixtb grade students were ea 
California Achievement Test. 
Level 15^ an cut-of-level te 
and Level^ The restilts ar 
concludes that out-of-level 
limited solution or stop.gap 
problems in Title I evaluati 



expressed for the ^possible effects of 

Education Act (ESEA) Tit Je I students 
hat may be so difficult that many' 

the chance range* The likelihood of such 
led scores if they ^were tested with 
s discussed. In this study, two groups of 
bh tested with two levels of the 

One group took Level 16, cn-level, and 
st,^ while the ot^ier group took Level 16 * 
e presented in tables- The author " 
testing of low achievers is, at best, a 
measure to £h€ pressing instrumentation 
on. (CTM) X " 



1- 



]^^:^jic:^:^]^«««:»««««1i^««««4(« ♦♦♦♦♦♦ ♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦>f^ 

♦ Reproductions supplied by EDRS are the best that can be made | 

♦ from the original document* ♦ 



ERLC 



a 



^ 4 

wm Out-of-Leve1 Norni-Referenced Testing Improve the Selection of 
Program Participants and the Diagnosis of\^Readinq 
Comprehension in ESEA Title I Programs? 

standardized norm-referenced tfesting programs have geneVally ' 

^ posed some serious problems for populations that tend 'to score far 

below the Inean of the national norm group. Te|t results appear to be 

less meaningful, and student-teacher frustration increased, as large 

groups of low-achieving students receive scores that approach the chance 

level on norm-referenced tests. T^t company representatives have 

suggested and the RMC Research Conporation has recommended that ' 

out-of-level testing of students in schools containing large numbers 

of low achievers would provide a solution to this pressing instrumenta- 

'tior> problem, and improve the evaluation of programs designed for 

low-achieving groups such as ESEA Title I. Both the test companies 

and the RMC Res^earch Corporation have' assumed that tests providing 

an expanded standard score scale will permit conversions of students' 

out-of-level raw scores to in-level percentile norms and that the 

students' percentile rank (or NCE) will be essentially the same as . 

would have been obtained if the appropriate grade-level test had been' 

'.used (Roberts, 1976.)'. - * 

«r , The major purpo'sfe of this study 'was to test , the assumption 

that groups of .students in Jow-achieving schools would indeed obtain 

* ' '•' ' 
the same interpretive score, e.g., percentile rank and NCE, when 

''^ ^ , . ' ■ ^ . ^ " 

testing on reading tests that were designed for studen1;s one or more 

* ^ * < * , . 

years below their grade placement level as they would have obtained on 



grade level tests. After exploring the main effect of out-of-level 
^testing upon mean NCE level, the implications of out-of-level testing 
for selection .of TitleM program participants' and for improv^ the . 
assessment of student achievement for very low proficiency readers were 
determined. 



Method 

The 1977 edition of the California Achievement Test, Form C 
(CAT/C), levels 14, 15 and 16, Reading Comprehension subtest, was used 
to test sixth grade students in Title I schools. Each student took the 
on-level test, CAT/C, level 16,' designed, for. grades 5.5 to 6'9 and one 
of two below-level tesj:s, level 15 designed for^gv^ades 4.5-5.9 or 
level 14 designed for grades 3.5-4.9. The CAT/C was especially designed 
for functional level testing in that locator tests were provided for 
selection of the appropriate test level and Instructions were written 
^ that multiple levels of the test could be administered concurrently. 

A balanced testing design was used in which half of the 

students in each of the four classes took' the on^level test first 

anci the off-level test tWo days later; the prqcpduVe was reversed for 

the other half of the group. Research Group A ;(N=4/tt took- the on-levql 

test and the one-level -below-level test (CAT/C 15). psearch Group B 

(N=54) took the on-level test and a test two^levels-t/elow Tevel 

(CAT/C 14). ' ^ , 

> 

Students vere given CAT/C LocatorJTest 1 during the week . 




previous to testing. Locator test scores ..were not used to assign test 
levels but were used to explore implications r^aarding numbers^ of 
Title I students who would be assigned .lower lev6l tests on this basis. 




and to further study the ramifications ' of ^individualized testing for 
Title. I populations. (A subsidiary study of individualized testing 

rried out at three *other schools using locator test scores and 
teacher judgment indicated that teachers invariably based their 
selection of CAT/C test level upon locator test scores.) 

Results and Discussion • ' » 

Since there were no order effects for either of the research 
groups, data for students tested either first or second on the in-and- 
out-of -level tests were combined. • 

The results shown in Table 1 indicate that low-achieving 
students tested two-levels-below grade level obtained statistically 
significant (p<.001) lower'standard scalp and NCE scores than they 
obwdined from on-level tests. Students tested one-level-below grade 
level also received lower norm-referenced test (NRT) scores on the 
off -level test than they obtained on the pn-level test but the 
difference was not statistically significant. 'It is mentioned here 
because*^i4: suggests a systematic trend of decreasing NRT scores 
accompanying an increase in the number of levels below-grade-level at 
which students are tested. This may be especially important since 
Research Group B, taking CAT/C levels 14 and 16, was a lower achieving 
group than Resea^rch Group A and issues of the effects of functionBl- 
level testing are especially pertinent to assessing 'the achievement of 
that sector of the population. "As^was expected, students in both 
research groups achieved significantly higher raw scores on the below- 
Iftvel tests than from the on-level tests indicating that the lower level 
tests were. indeed easier for most students. Correlations between test 
, levels were significantly high: 



l<\ interpreting these results, it should sbe noted that the 
RMC Research Corporation (Roberts, 1976) recomniGnds out-of-level 
testing for low-achieving groups whose rn^n is about a third of a 
standard deviation higher than the median even if the average raw 
score of a group is above chance level as were both .research groups 
here. For the combined research greups (N=101), the raw score mean 
(M=18,35) for CAT/C 16 was approximately one fourth of a standard 
deviation (^D=8.25) highe^ than the median (t!Wn=16.29) . This would 
suggest that some students were experiencing floor effects but not, 
quite of the magnitude suggested by Roberts for out-of-level testing. 



Insert Table 1 about here 



. ; The RMC technical paper also suggested that it waL&^erally 
only necessary to drop one test level in out-of-level testing to 
"improve" the tesfing situation for low-achieving groups. We examined 
this p/esupposition in a number of way s"^ in our study. Firlt we used 
the CAT/C lx)cator test scores to estimate the .number and percentage of 
Title I students that would' have been assigned to lower-level tests 
in an^lndividualized functional-level testing program. As indicated 
in Table 2, data regarding test levels prescribed by Locator Test 1 
would overwhelmingly select below-l-evel tests for this particular 
sample of Title I students*. /The results show that 57 percent >fould ^ 
have been* tested , at two or m6re levels below grade level and , as a whole 
when the 15 percent prescribed at level 15 were Included, -73 percent 



of* the group would have been tested below level 



Insert Table 2 about here 
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The question that arises immediately, is whether or no't such 
extensive out-of -level testfng would 'liave been warranted. For this 
purpo|e the data were blocked and analyzed separately for students at . 
three proficiency levels on each Jevel of the. test. ^ Students were 
'judged at floor level if they obtairfbd a^score that could have been 
obtained by chance, i.e.,<10 or fewer raw score points out of 40 ^ 
po^stVle' points. Our definition of floor level may be \ovt since some 
measurement experts, e.g., Gulliksen ^(1950, p.^'zes)., would, consider 
scores wjthin one or even two standard* deviations of a chance score 
to also be at floor level. Students were considered in-range if they 
obtained a raw score of 11 to 30. Ceiling effects were defined ^s . 
31 or more raw score points. Table 3 displays NCE means for pro'ftciency 
groups defined tor both the in-level and out-of-level tests. Uable^A 
in ffie appendix presents the standard score data.) Eight students in 
Research Group A and fourteen students inResearch Group B, or a total 
of 22 percent were at the floor^leve*l on the CAT/C level 16 test. 
There is a tremendoi/s difference between 22 percent in need of easier 
tests based upon empirical results and the 73 percent predicted by , the 
locator test. * ' i - . 



Insert* Table 3 about here 



. --^-^ . ^ 

^ TJiere has been some^peculation that lower norm-referenced 

scores received by low-achieving groups on off-level tests may more" 

truly reflect the acWevement levels of Title I students. We disagree 

with that prop'b&itie'". The* data presented in Tables ^ and 4 indi(;ate 

♦ 

that students who received in-range scores on one or both level s^^f^ 

0 / 

the test received higher ^nomi-referenced scores, e.g., NCEs or^ • 



percentiles, for"* on-level tests. Students scoring in-range on level 16 

received a mean of 5.9 NCEs'higher (p<:.001) for CAT/C. level If than 

they received on level, 14. 

. Students- scoring^in-range, and at the floor level on both test 

levels 14 and 16 also received signififcantly higher NCE means for the 

on-level ^ test (Table 4). The other proficiency group means were also 

higher for the ojiJevei test Wit differences were not statistically 

significant^. It would seem important in current nationwide -focus upon 

low achievement not to f yrther Vigment the position o-f groups who 

disparage the achievement of today's students. ' 

i 

Insert Table 4 about here - \ 



The number of students at floor level on the off-level test 
was not remarkably differerlt than that ofv the on-ievel jtest. Nineteen 
students or three less than for the on-level test also score'cl at th*e 
chance Tevel on the below-level tests ! There was even an increase of 
one fl^or level score for CAT/C level 15 over 'level 16. - (This can 
' easily have occurred by chance.) Therefore the data presented in ' 

'V . , • s ' • 

Table 3 suggest that testing vary low-proficiency readers one or even 

' * \ * 

two tesf'levels below grade placement does not ^olve the problem of q 

ft 

/.providing adequate instrumentation for tejtirig the reading performance 

of^,t1ie 19 "to 20 percent of students in the^vefy low-achieving group. 

To further explore this question! the number of students sporing. 

. ^ * r- 

the same proficiency level on both tn-'^nd off-level tests, espepially 

those at floor level, was ecamined. Table 4 ^indicates that 5 students. 

or IVpercent in group A were at floor level on txoth tests 'and 7 students 

or 13 percent in group B were at floor level on both tests. Probably 



some degree of test unreliability, often/mentioned in colfhection with 
low-achievers, combined with exact criterion levels accounts for the ' 
reduction from 20 percent 5f one test score to 12v percent on two test 
scores being at chance or floor leveT. (Table B in the appendix 
presents the standard score data.) As seeo in Table 5-^ few students 
in-range on either test scored at the floor on the other regardless qf 
test difficulty. / ~ , 



Insert Table 5 about here 
- - _ ^ « \ 
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In order to more clearly understand the' phenomenon of "^elow 
chance level" achievement test scores, the locator test scores of ^ 
students scoririg at floor level on" level 16 or on both levels were 
examined. /\11 students at floor level on level 16 had received locator 
test placements Indicating a lower level of the CAT/C was needed. All 
studeats at floor level on both^tests had received locator test 
placements of at least one level below the lowest level of the test 
they had taken. Of the five students at floor level on both levels 15 ' 
and 16, one received a locator test placement of 12, three were at- . 
level 13 and one was at level 14. Of the six of ^seven students at 
floor level on levels 14 and 16 for whom locator test scores were 
available, five placed at level 13 and one at levef 12 on the locator 
test. *0f course, of 32 students receiving locator test placements of \ 
12 or 13, only 12 students actually required tests this far removed ^ 
from grade placement level. ^ 

"' \ 

After examining locator test scores, the Title I reading 
teachers were questioned regarding the characteristics of the floor- 
level group. Five of the twelve students who scored at^'the chance 

, ■ f" 
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level on both tests were classified as learninc) disability children. 

Tho readincj ability .of the remaining students was described as low third 

grade or end^ of second grade instructipnal level and their behavior was 

*descri4)ed as immature or needing a great deal of guidance in the 

reading process. One child had aa unusually limited. language and 

experiential background. Only one child; described as a ^ery good 

thinker, came clDse tp the category 6f non-reader. The reading profi-. 

ciency of students who- had been at floor on-level Hut in-^range p?'f-level 

was judged somewhat higher by- the teachers. (Two^ of these ten 

children were also' .learning disabi-lity children and one other was 

making a rapid transition from Spanish to English reading.) In 

February, teachers tested five children on the level 12 and 13 tests - 

and ali but one learning disability student received in-range test 

• » • , • \ 

scores.* It seems that CAT/C, level 13, designed for grade placement 
2.5-3.9 may match the students' reading proficiency level according to 
both teacher judgment and actual test performance. 

Table 6 displays data showing' the shifts occurring between * 
percentages of students scoring at various proficiency -levels with * 
out-of-level testing. It appears that there is a greater shift 
towards' more ceil|ng effects with out-of-level testing^than there is 
a decrease in the percentage of floor effects. Further, only 69 percent 
of the group scored at the same proficiency level on both, tests. 



Insert Table 6 about here 
*""^"""s """"""""" 

. In terms of stanine,, NCE and percentile rank equivalents of 
raw score intervals for on- and off-level tests,' Table 7 presents data 
indicating that a greater probability, i'.e. , greater stanine range, of 



ceiling than floor effects can be expected for all levels on this 
p^r1|MjKjUr subtest. For exapiple, chance raw scores extend into 
stanine wo for level 16 while ceiling effects include stanines seven 
througb nine. (By tbd end pf grade'sixy ceiling* effects extend into 
stanine six.) Chance raw scores for level 14, two levels'below grade 
placement, are confined within stanine one wKile .ceiling effects - 
include stanines five through nine. This shows the magnitude pf raw 
score points required to obtain a simi.lar percentile rank on a^lower 
level test than would be required on an on-level test. 



Insert Table 7 about here 



Students* are a9mie.times selected for participation in Title I 
programs on the basis of standardized test scores. A stanine range, - 
of 1-3 is a common criterion for defining the Title I target population. • 
The data present, in Table 8 show that if students are tested oneMevel 
below grade placement th^e will be an increase of six percent in. the 
target population. "The increase for Research Group A was from 28 percent 
for on-level testing ^to 34 percent for off-level testing. In Res^earch 
Group B, 57 percent 'of students tested two levels below grade placement 
would be selected as t£|rget students while only 46 percent .were 
selected when tested on-level. Therefore, testing two levels belo^y 
restated in increasing' the target group by' 11 percent. 



Insert Table 8 about 'here 



The combined research groups increased the target- population • 
by 9 percent when testing one or two levels below level, an increase 
.from 38 percent on-level to 47 percent off-level. ' Although groups A%and B 

11 
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were not equal, they are both part of the eligible target school popu- 
lation in one4istrict. If an individyalized'^testing program were 
imp^enented ratherJ|^an a controlled design, a tai;ge%-^)opu1ation 
selected upon the basis of test scores would be increased relative to 
the numbers of lower level -tests administered. ^ 

.Conclusions and Recommend ations-^ .^^ ^ ^ 

Some previous research comparing the out-of-level test scores 
of TitTe I students in Rhodeilsland with on-level test scores found 
that different grade-equi val4nt scores were^obtain^ed for off-level and 
oh-level tests (Long, Schaffran and Kellogg, 1977). Grade-equivalent . 
scores were lower for grade four students when tested below-level and 
higher for grades two and three students when tested below-level.' The 
findings' ©f this study agree with the .Rhode Island fourth gride results^ 
in that sixth grade students in the T&cspn Unified School District * 
.Title I program obtained lower derived scores,, i'.eJMCEs , when tested 



ivei. 



below level than .when tested at -grade placement level. WMle that 

study used the Gates-MacGinitie ReadringcJest and this one u?ed the . * 

CAT/C Readihg Comprehension subtest both studies suggest a need to' 

reexamine .the procedures u§ed to develops norms for tests to be used at 

both grade and functional level. Rather^ than relying <solely upon'over- 

lapping each level of the CAT/C in the spring with^he next higher t6st 

' Jit* - ' U 

level in constructing Jthe expanded^ standard score scales, enipil^ical 

norms should be* developed for -out-of-level testing by testing -students 

at multiple tes.t levels in. the norming procedure. . If a systematfc ' 

bias to derived scores;of out-of-level ^ests re^nains a problem, perhaps 

a regression formula could bV'used ta provide an^estimated in-level^ 
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derived score. As stated by Long'et>al., (V^, p. 212), "Clearly 
this study indicates weaknesses in cross-form scaling- of the test 
used." . . ' 

The issue of test fairness to low-achieving students -is 
becoming increasingly important as more political pressure is put upon 
educators to maintain minimum standards in , basic sJcill areas. As was 
found in this study and in the Rhode Island study, even larger percent- 
ages of students would bfe selec^ted for remedial-type programs, on the 
basis of off-level tests than on the[b.asis of grade-level tests. It 
would seem that the last thing- a "Ttow achiever needs is at test that is 
even more difficult to " pas s"| than those currently in use. ' - 

A reasonable plan for Title I ^sting would seem to be to 
test mbst students on-level and to retest only students scoring below 
chance level on locator tests and the prescribed lower-level test.* 
However, the data from on- ait^ off-level stests should be kept separate, 
for both individual and group evaluation purposes- If a test were 

adminisi^red one. level below level, the use of a percentile band for 

.4 ^. - ' ' ^ 

individual scores would probably suffice; however the actual need > 

seem^ to be ^for^'tests two or more levels below grade placement for' 
'the^very low proficiency, reader. (The CAT/C has more refined levels 
than other test t>atteries so the number of levels below" grade placement 
used will be dependent "upon the test battery.)' Also, the effe^s of ■ 
■out-.of -level testing will have to be researched Tor each' different^grade 
level and test battery.. , : ^ j 

The value of out-of-level testing is that it is one way to 
measure the achievement" of some very low proficiency readers rather 
than to treat all low achievenjjpFas a void. It is important to obtain 

.13 ■ 



in-range scores for as lartge a number of students as possible. Yet, . 
-vre must question the adequacy of the lower levels of the CAT/C-^as a 
criterion-referenced measure of the reading proficiency of older 'low 
achievers. The term functional level testing-has been avoided in 
connection vhith this study as the CAT/C and. other survey tests may 
not be .comprehensive enough to adequately measure functional reading 
proficiency, especially for students at increasingly higher *gfade 
levels. In fact, there is probably a trade-off between how well a 
lower-level test serves as an appropriate, measure of functional reading 
performance and the congruence of derived off- and on-level test 
scores - . Reading is measured in a somewhat different-way in the CAT/C 
level 12 and 13 tests than in level 16. ^t the lower levels there are 
fewer comprehension items following passages and sentence length 
comprehension, structural analysis and word recognition items are ''^ 
included. The tests' also require a guided testing technique where oral 
directions are frequently giveo during the testing sessions. Therefore, 
the levels 12 ^nd 13 approach to testing seems* appropriate to reading 
behaviors of "chance level" students as described by the reading 
teachers.' 

It appears that out-of-level testing of low-achievers is at 
best a limited solution or stopgap measure to the pressing instrumenta- 
tion problems in Title I evaluation.. One isi reminded of the dilemma 
of a chef who 'has; a saucepan'without a handle which s/he will not throw 
away! for l£ick,of a replacement. Y6t the saucepan is very awkward to 
use. pne can use it but it is tricky. That is the situation at present 
for anyone trying to interpret test results from out-of-level tests. 

V ' ^ •• 14 
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Table 1. Conipari'son of Reading Achievement Scores of Sixth Grade Students Tested On and Below 
Grade Levej on the CAT/C ' 



RS 

M SD 



ss 



SD 



Group A 

Level 15 
Level 16 
Di f ference 



Group B 

Level 14 
Level 16 
Di f ference 



Correlations 



47 
47 i 



22.3 
20.7 
-1.6* 



10.1 
8.6 



475.2 
481.3 
6.1 



71.9 
63.0 



54 
54 



20.0 
16.3 
-3.7*** 



9.1 
7-. 4 



435.1 
452.3 
17.2*** 



58.4 
55.2 



Levels 15 and 16 
Levels 14 and 16 



. 88*** 
. 86*** 



,89*** 
,83*** 




Percentile Equivalents 
of NCE Means 



.84*** 

/83*** 



38.4 
43.2 
4.8 



ia.5 

25.9 

7i4 



1 



Mote ; 



Correlated t tests were used to compare the significance of the difference 
between means for all but the percentile equivalents of NCEs. 



* p < .05 
** p <. .001 



Table -2. CAT/C Test Levels Prescribed by Locator Test 1 





Group . 


■ 12' 




. CAT/C Leverls , 








13 


14 15 • . - 16 , 


17 


Total 




M % 


N 


N % % N % 




N* 



A 


2 5^ 


.9 


21 


11 


25 


o 


3 6 


13 


30 


9 




Total 


5 6 


27 


X20 


32 



19 


10 


23 


'3 


1 


43 


12 


11 


24 


1 


2 


^48 


15 


21 


23 


^ 


- 4 ■ 


91 



a 



* Some students were absent on tbs clay the locator test was 
administered 
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Table 3. Comparison of NCE Means for Groups^^locked on Floor, 
' Ceilinq and In-lRange Proficiency 1/e^^els and In-and- 

Out-of -Level Tests 



Group A Proficiency Groups Via CAT/C Level 16 



Test 



Floor 



In-Rarige 



Ceil ing 



Total 





• . N 


M 


N M 


N 


H 


N 


M 


CAT/C 15 
CAT/C 16 


8 
8 


17.6 
15.6 • 


3^ 44.1 
32 47.1 


7 
7 


75.0 
77.4' 


47 
47 


44.2 
46.2 


Difference 




-2.0 


3.0 




2.4 




2.0 


Group A 


Proficiency Groups Via 
•ft 


CAT/C 


Level 15 






CAT/C 15 
CAT/C 16 


I 

9 
.9 


14.3 
' 22.3 


25 40.5 , 
25 43.5 


13 
13 


71.9 
68.2 


47 
47 


44.2 

/46^2 


Difference 




/ 


3.0 




3.7 




2.0 



Group B Preficiency Groups Via CAT/C Level 16 



Test 

CAi7c''i4 

, CAT/C- 1&, ' 

Difference 
1 



CAT/C 14 
CAT/C 16. . 

Difference 



Fl oo.r - 
N M. 

14 T4v2' 
14 16,4 

2.2 



In-Range 
N M 

37 34.9 
37 40.8 



5.9*** 



'Ceiling 

N M ; 

3-'' 63.9 
3 75.6 

' 11.7 



•Group B Proficiency Groups Via CAT/C Level 14 

10 6;2 35 31.2 9 ^58.8 / 

10 21.5 35 34.2 9 61.3 / 



15.3** 



- 3.0 



Total 
N M 

54 31 .1 
54 36.4 



5.3*** 



.54 31.1 
54 36.4 



5.3*** 



** p < .01 
*** p < .001, 



^ble^4V Comparison 6f NCE Means for Groups of Grade Six Students 

Scoring at the Same Proficiency Levels on CAT/C Levels 
1 • ' ,14, 15. and 16 



Qroup A" (N=34) i 
Pno/iciency Groups 



One-l^evel -Below 
CAT/C 15 
N M. . ^SD . 



On-Level 
CAT/C 16 
M SD 



Difference 



Floor 1 
In-Range 
Celling \ 

\ 1 


5 11.9 7.6 
22 42.3 9.2 % 
7 75.0 10.7 


13.6 rS.O 
46.8 12.1 
77.4 5.1 


1.7 
4.5 


Group B (N=36) ' 
Proficiency IGroups 


Two-Level s-Below 
CAT/C 14 
N M SD • 


On-Level 
CAT/C 16 
•M SD 


« 

D1 f f erence 


Floor 1 ■ 
In-Range 1 \ 
Ceiling 1 


7 6.2 5.2 
27 32.9 7.9 
2 72.5 17.1 


16.5 5.1. 
37.5. 7.5 
78:3' 16.0 


10.3* 
4.6**?^ 

5;fr 

I 


* p < .05 1 








*** p < .001 1 









Table 5. Comparison of NCE M&ans for Students at Floor and 
In-Range on In-and-Off Level Tests 



Group A 

Floor on 16, In-Range oi\ 15 
FT«<|r on 15, In-Range on 16 



3 
4 



CAT/15 
M SD 

27.1 3.3 
17.3 -.- 



CAT/ 16 
M SD 



18.9 
33.2 



5.0 
5.4 



IDif Terence 

-8.2 
15.9 



Group B 

Floor on 16, In-Range on 1*4 
Floor on 14,. In-Range onH6 



7 

3 



Cat/14 

M . SD 

22.1 5.2 
6.1 4.8 



CAT/16 
m SD 



1,6.4 
33,2, 



7.9 
2.9 



Di ff erence 

. -5.7 
27.1 



f 

19 



Table 6. Numb.er and Percentage of 101 Students at Proficiency 
Levels on In-Level, Out-of-Levfel, and Both Levels of 
the CAT/C - , . 





Both Levels 

■ N 1 ■ 


In- 
N 


•Level 

% 


Out-of- 
N 


•Level 

« 


In-Range 


• 49 


49 • 


69 


68 

r 


60 


59 


Ceiling 


. 9 


- 9 


10 


10 


22 


22 


Floor 


J2 


12 


22 


22 


19 


19 


Total ' 


70 


69 


101 


100 


101 


100 
f 



^5? 
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Table 7. Raw Score Cutoff Points for Stanines, Percentile Rank, and Nearest Whole NCE /or Grade ^.1 Students 
for CAT/C, Levels 14, IS and 16, Reading Comprehension-'Slibtest^ 



Stanine, Percentile Rank and NCE • ° , - 

StaninS 1 2 3 -A ' ^S 6 7. - -8. -9 

Percentile Rank 1-4 ^".5-10 11-22 23-40 41-59 60-77 • ' 78-^89 90-95^* ^$6-99 

NCE ' 1-13 " ' 15-23' 24-34- "34-45 •45-55^ . 55-66 " ' 66-76 ■77-85 87-99 

RS . 0- S* ^9-10* ll-14j 15-20 ■ 21-25 -26-30. 31-34** 35-36** '37-40** 



CAT/t 15 



4^. 



RS 0- 9* 10-11* 12-16 17-23 24'-29 30-34** 35-36** 37-38** 39-40** 

CAT/C 14 ' ' * ' 

RS. 0-10* n-14 15-21 22-28 29-33** 34-36** , -37-3'g*'*'- ; 39** ^ 40** 

^Note: Off-leVel raw. scor^s^ere converted to^off-level standard sqores which were conveVtedAo on-level 

percentiles for grade 6.1.^^ ' 

^Chante level raw score range l-IO. ^ - . ^ * . ' 

7**Cei ling -level ray score range 31-40. ' , 



fable 8. ■ Size of Target Groups, Stanines JUSr-tor Students 
Tested at Nominal level and-Onror Two levels 
Below riom4fla4-L-ev€-l 



Research Group A 

Total Group tested 

On Level (CAT/C, 16) target group 

One Level Below (CAT/C, 15) target 'group 

Difference • 

Research Group B 

• Total tJroup tested 

On Level (CAT/C, 16) target group 

Two Levels below (CAT/C, 14) target group 

Difference 

Combined Research Groups 

Total Group tested 

'On Level (CAT/C, 16),- target group 
Off LevdS, (CAT/C, 15 & 14) target group 
l^fferenc'e 



N 

47. 

13 
16 



54 

25 

.31 



101' 

38 
47 



28 
34 
6 



46 
57 
11 



38' 
47 
9 



\ 



9 
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Table A. Comparison of Standard Score Means for Groups Blocked on Floor, 
Ceiling and In-Range Ptoficiency Levels on Ih-and-Out-df-Level 
Tests. 

Group A ' [ [ 

Tests J?.roficiency Groups Via Level 16 Test 

Floor , In-Range * Celling -Total 



CAT/C 15 


H 


M 


N 


M 


n: m 


' N' 


M 


8 


387.9 


• 32 


474.9 


7 576.0 


47 


475.2 


CAT/e 16 


8 


387.5 


32 


482.4 


7- 584.1 


47 


481.4 


Diff. 




. -.4 




7.5 


. »r 8.1 




6.2 


CAT/C 15 






Proficiency Groups 


Via Level 15 Test 






9 


377.8 


25 


463.0 


13 ' 565.9 


47 


475.2 


CAT/C 16 


9 


408.2 


25 


470.1 


13 553.8 


47 


lf48U4 


Diff. 




30.4 




7.1 


-12.1 




6.2 



Group B 

Tests ] Proficiency Groups Via Level 16 Test 



Floor f \^ In-Range Ceiling Total 

M_ \Aj M_ N " M N -M 

CAT/C 14 ' 14 379,6 441.7 -3 539.7 54 435.1- 

CAT/C 16 14 38715 ^7 46^;7 3 577.7 54 452..3. 

Diff. 7.9 L9.0 38. p,.^ 17.2 

Proficiency Groups Via Level 14 Test 

CAT/C 14 10 352.3 ^5 435.7 9 524.9 " 54 435.1 

CAT/C 16 10 403.6 35 445.8 9 531.8 $4 452.3 

Diff. 51.3 , 10.1 6.9 17.2 



, 25' 



i. 



Table B. Comparison of Standard Scores For Proficiency Groups of Students bn 
Two Test Levels. 









Below Level 




CAT/16 


Diff . 




Group A (15/15) 
Proficiency Groups 




N 


M 


SD- 


K ■ 


SD 

* 


* 




Floor, -Both Levels . • 
. In-Range, Both Levels 
Ceiling, Both Levels ' 




22 
7 


367.2 
468.6 
576.0 


34.9 
29.0 
35.5 


- 381.2 
480.0 
584.1" 


23.1 
28.5 
17.0 


' "14.0 
^ 11.4 
8.1 




Floor on 16, .In-Range' on 
Floor on 15 , Iii-Range on 


15 . 
16 


3 

• 4 


' 422.3 • 
391?a 


11.7 


398.0 ''^ 
442.0 


15.6 
17.3 


• -24.5 
-> 51.0 





Group B (14/16) 
Proficiency Groups 

Floor, Both Levels 
In-Range, Both Levels 
Ceiling, Both Levels'^ 





7 


- 352.1 


16^ 




^ 27 


441.3 


.24.9 




2 


566.5 


54.4 


14' 


* 7 


407.0 


16.8- 


16 


3 ^ 


352.7 


11.5 



386.8, 
, 456.6 
'^87.5 

388.1 
442. 7L_ 



/ 



18.7 
23.4 

54'! 4 

24.6 
8.1 



4:5.3 
21.0 - 

-;8.9 
90.0 
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